Learning Bayesian Belief Network Classifiers: Algorithms and System

This paper investigates the methods for learning Bayesian belief network (BN) based predictive models for classification. Our primary interests are in the unrestricted Bayesian network and Bayesian multi-net based classifiers. We present our algorithms for learning these classifiers and also the methods for fighting the overfitting problem. A natural method for feature subset selection is also studied. Using a set of standard classification problems, we empirically evaluate the performance of various BN based classifiers. The results show that the proposed BN and Bayes multi-net classifiers are competitive with (or superior to) the best known classifiers, based on both BN and other formalisms; and that the computational time for learning and using these classifiers is relatively small. We also briefly introduce our BN classifier learning system – BN PowerPredictor. We argue that BN based classifiers deserve more attention in the data mining community.